Repository navigation
Declarative calls - #5264
Declarative calls#5264
Conversation
c239e44 to
39c161b
Compare
66498aa to
10e4d8d
Compare
| } | ||
| } | ||
|
|
||
| fn check_block(&self, block: &BlockPtr) { |
There was a problem hiding this comment.
I think instead of "check_block" something like "clear_block_if_different" or something would make it more clear while reading callsites
| for handler in &self.mapping.event_handlers { | ||
| for call in handler.calls.decls.as_ref() { | ||
| match self.mapping.find_abi(&call.expr.abi) { | ||
| // TODO: Handle overloaded functions by passing a signature |
There was a problem hiding this comment.
Is this something you will address on the next PR or would it be better to add an issue for it ?
| /// ...)` where the `address` and the args are either `event.address` or | ||
| /// `event.params.<name>`. | ||
| /// | ||
| /// The parser is pretty awful as it generates error messages that aren't |
| } | ||
| } | ||
|
|
||
| pub async fn process_trigger( |
fcf9aab to
18e991b
Compare
|
Using a reliable eth provider, this ran cleanly in the integration cluster with no PoI divergences (test ran for 3 days) |
Remove a bunch of intermediate functions to call process_trigger_in_hosts directly
The loop over 'normal' events is broken into distinct steps for match_and_decode and for process_block
We need to make sure that the result of an eth call is entered into the in-memory cache synchronously, before handlers are run to ensure that they will find the call result. Only the part that writes it to the store can happen asynchronously.
Log eth calls in a form that's easy to plug into other tools (like foundry's cast) Demote the more internal logging of encoded eth call data to trace
This also renames some of them, but has no functional change
We need to relate requests to EthereumCalls that we were given even when the requests are reshuffled from the initial calls
This tries to avoid having to return errors for each call separately from contract_calls, while preserving the distinction between errors that indicate reorgs nad other errors.
The DecoderHook now relies on EthereumAdapter's contract_calls method. With that, cached call results are looked up with one query from the store. Call results that are not cached are retrieved with eth_calls made in parallel to the ethereum node.
The DecoderHook and the RuntimeAdapter for ethereum need to share the same call cache, a separate one for each subgraph, since the BufferedCallCache needs to be a different one for each subgraph - if subgraphs shared a BufferedCallCache, they would clobber each others declared calls.
|
Hi @lutter This looks like an useful feature that could increase our subgraph indexing speed a ton. So thanks! Question, if a declarative call is defined in the manifest, are these RPC calls always executed eagerly, regardless if the handler actually called it? The reason I ask this is because some of our handlers have conditional logic to decide whether to execute the RPC call or not. I'm just wondering if I'm now in the position of choosing whether to optimize the RPC calls' performance or the number of RPC calls. |
|
Link to "declared |
Yes, the declared calls are made no matter what. Whether declared calls in your situation will speed things up or not will depend on how likely it is that a call is made/not made, but since declared calls are all made in parallel, the cost for unnecessary calls (within reason) is very small |
Addresses #5262
This PR is concerned with two things:
The results of
eth_callsare injected back into the mappings via aBufferedCallCachewhich keeps the call results for the current block in memory. That way, mapping code does not need to change, and users can reap the benefit of declarative calls simply by, well, declaring them in the manifest. There's a small overhead in this approach as it requires calling a host function; compared to making an actual RPC call or just a database lookup, that's negligible.When we do eth calls in bulk, we do one query against the store for all the calls we need (rather than one per call) For the calls for which we do not have anything in the database, we make RPC calls in parallel. By declaring eth calls, the time for all calls goes from the sum of the time the calls take to the max. Depending on the number of calls a subgraph makes for a block, that can be a considerable speedup. For one subgraph which did 10-20 eth calls per block, the time to process a block went from ~ 1.2s to ~ 200ms, a sixfold speedup. The environment where this was tested took 70-80ms per call.
Done:
eth_callfailsStill todo:
eth_callsfor other handlers, not just event handlers. ([Feature] Allow declaring eth calls for call handlers #5330)This PR is already pretty big; I will file the remaining open items as separate issues before merging this.